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ABSTRACT 

The Web Audio Evaluation Tool is an open-source, browser- 
based framework for creating and conducting listening tests. 
It allows remote deployment, GUI-guided setup, and analy- 
sis in the browser. While currently being used for listening 
tests in various fields, it was initially developed specifically 
for the study of music production practices. In this work, 
we highlight some of the features that facilitate evaluation 
of such content. 


1. INTRODUCTION 


Perceptual evaluation of audio is an important part of re- 
search in virtually every audio-related field, and music pro- 
duction research is no exception. To facilitate listening test 
design without time-intensive development, compatibility 
issues, and limited interface options, the Web Audio Eval- 
uation Tool was developed [ 1 ]. Initially developed with the 


evaluation of mixes in mind |2], the tool has since evolved to 
an all-round listening test platform, used and shaped by re- 
searchers from a wide range of audio research topics. Some 
prominent characteristics relevant to intelligent music pro- 
duction are discussed here. 


2. FEATURES 

In many cases, tests are run on one or more dedicated com- 
puters in listening rooms, sometimes in different countries. 
As these computers may have different operating systems or 
supporting software, developing an interface that works on 
all machines can be a challenge. This led to the choice for 
a browser-based tool, which only needs a browser that sup- 
ports the Web Audio API and is therefore platform-indepen- 
dent. Equally important is the opportunity for ‘remote’ tests 
this offers, meaning no installation or setup of the tool is 
required, but participants can simply go to a website. 

The comparison of differently processed musical sig- 
nals is further made possible by instantaneous switching be- 
tween stimuli, and synchronised playback of time-aligned 
audio fragments. This leads to seamless transitions where 
the relevant sonic characteristics change immediately while 
the source signal seemingly continues to play, and avoids 
excessive focus on the first few seconds of long fragments. 


This is especially useful for the comparison of different sig- 
nal processing algorithms or parameters. Optional cross- 
fading or inter- fragment silence accommodates other types 
of tests. 

Whereas most available software still requires a sub- 
stantial amount of programming or tedious configuration on 
behalf of the user, the Web Audio Evaluation Tool allows 
anything from test setup to visualisation of results to happen 
entirely in the browser, making it attractive to researchers 
with less technical backgrounds as well. The code or test 
configuration files only need to be altered when advanced 
modifications need to be made. Even for users proficient in 
web design, the GUI allows very quick setup and an imme- 
diate overview of test results, see Figure |T] 
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Figure 1 : The test creation GUI allows quick and easy set 
up of any but the most advanced and customised interfaces 

Based on experience from several researchers working 
on different topics, various other features have been added 
to address common difficulties and issues when designing, 
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conducting and analysing tests. Surveys are integrated in 
the interfaces so that answers are collected immediately be- 
fore or after the test, or even before or after specific pages 
in the test. These include validation of responses and vari- 
ous question formats such as checkboxes, radio buttons, and 
embedded videos. 

The test timeline and ratings can be visualised imme- 
diately upon finishing the test, to spot errors or misunder- 
standings quickly, or even do basic analysis of the results 
thus far in the browser, see Figures [2] and [3] 



Figure 2: Timeline for a specific subject and test page, 
showing playback of the stimuli (red) and movement of the 
associated sliders as a function of time 
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Figure 3: Box plot visualisation of results from several sub- 
jects 


3. INTERFACES 


Every common interface type for perceptual evaluation is 
supported by the Web Audio Evaluation Tool, and each can 
be customised to a large extent. For a modest number of 
stimuli, a multiple stimulus interface is preferred over sin- 


gle stimulus or pairwise comparison [3 4 ] . As there is usu- 


ally no reference when comparing differently processed ver- 
sions of a musical signal, the popular MUSHRA interface 
1 5 1 may not be appropriate. Furthermore, modified versions 
of MUSHRA featuring a single rating axis with multiple 
markers have been shown to be more accessible [6], akin to 
the APE-style interface [7] which typically includes com- 
ment boxes as well - see Figure]?] 
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Figure 4: Example multi- stimulus test interface 


4. CONCLUDING REMARKS 

The Web Audio Evaluation Tool is a versatile, browser- 
based listening test platform, currently used for various ap- 
plications including the evaluation of music production prac- 
tices and audio processing algorithms. Apart from the time 
saved by using an off-the-shelf, feature-rich tool, research 
also benefits from an experimental apparatus that is well- 
documented and widely used. 

The authors highly welcome any feedback and contri- 
butions on the GitHub pag^ Source code is available as a 
ZIP or through git 1 2 or Mercuriaj^] 
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1 github.com/BrechtDeMan/WebAudioEvaluationTool 

2 code.soundsoftware.ac.uk/projects/webaudioevaluationtool 
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